THE PRODUCT OF MATRIX SUBSPACES 



MARKO HUHTANEN* 

Abstract. In factoring matrices into the product of two matrices operations are typically 
performed with elements restricted to matrix subspaces. Such modest structural assumptions are 
realistic, for example, in large scale computations. This paper is concerned with analyzing associated 
matrix geometries. Curvature of the product of two matrix subspaces is assessed. As an analogue 
of the internal Zappa-Szep product of a group, the notion of factorizable matrix subspace arises. 
Interpreted in this way, several classical instances are encompassed by this structure. The Craig- 
Sakamoto theorem fits naturally into this framework. 
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1. Introduction. This paper is concerned with the set consisting of matrix 
products of elements restricted to matrix subspaces Vi and V 2 of C nxn over C (or R). 
Matrix subspaces appear regularly in large scale computational problems where only 
[T 1 . modest structural assumptions can be made0 For example, factorization problems 

[551 EMI H3] are typically subspace problems. The study of matrix subspaces can be 
classified as being finite dimensional operator space theory. For operator space theory, 
see [HI US] ■ Regarding the geometry associated with matrix factoring [201 U j the set 
of products is defined as 



V1V2 = {ViV 2 : Vi 6 Vi and V 2 e V 2 }@ 

> 

£f} ' As illustrated by the LU factorization, both complete and incomplete, as well as the 

singular value decomposition, this structure is ubiquitous. (The subset of C" xn of 
matrices of rank k at most is also the product of two matrix subspaces of C" xn .) These 
examples also underscore how different aims and geometries the set of products can 
have. 

The set of products is not the most natural structure from the computational view- 
point of factoring!! It is encountered regularly, though. Through the LU factorization, 
band matrices is one such instance, even though it is exceptional by being a matrix 
subspace. In general, the set of products is constructible and certainly not flat. 
Thereby one is led to ask how curved V1V2 is. By locally inspecting the image of 
;_i ' the smooth map 



{Vl, V 2 )^V 1 V 2 , (1.1) 

this can be assessed Riemannian geometrically. This approach turns out to have a 
global character, leading to a necessary and sufficient condition for the flatness of the 
whole image of (jl.ip . 



* Department of Mathematics and Systems Analysis, Aalto University, Box 1100 FIN-02015, 
Finland, (Marko.HuhtanenOhut.fi). Supported by the Academy of Finland. 

x In large scale problems, a matrix subspace typically defined by fixing a sparsity structure with 
O(n) nonzero entries. 

2 In algebraic geometry, when Vi and V2 are treated as projective Hilbert spaces, the tensor 
product is the more usual object of interest (given by the Segre map). 

3 Being computationally far more accessible, it is likely that the set Vi Inv(V2) = {Vi V 2 T 1 : Vi £ 
Vi and V2 G V2 n GL(n, C)} is more important; see [20ll4]. 
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Vanishing curvature is intriguing by the fact that then the associated matrix sub- 
space factors, i.e., it is not a so-called irreducible matrix subspace. And conversely, it 
is a fundamental problem to recover whether a given matrix subspace is irreducible. 
Once interpreted this way, it becomes clear that this notion has many appearances 
already in the commutative case, such as integer factorization and polynomial fac- 
toring. In the noncommutative case of dim Vi = dim V2 = 2, an appropriate general 
treatment of the Craig-Sakamoto theorem in statistics is shown to correspond to such 
an instance. 

The paper is organized as follows. In Section [2] the product of matrix subspaces 
is defined. Closedness is addressed and some fundamental linear algebraic notions 
are recalled. The structure is illustrated with several examples. Through the Craig- 
Sakamoto theorem, special attention is paid to the three dimensional case. Bihomoge- 
neous polynomial maps of bidegree (1,1) are associated with the problem. In Section 
|3] the notion of factorizable matrix subspace is introduced. The curvature of the set 
of products is assessed locally. Based on this, a necessary and sufficient condition for 
the flatness of the set of products is given. 

2. The product of two matrix subspaces. The product of two matrix sub- 
spaces is defined and exemplifed. Special attention is paid to a generalization of the 
Craig-Sakamoto theorem. Bihomogeneous polynomial maps are associated with the 
product of two matrix subspaces. 

2.1. Matrix subspaces and the Craig-Sakamoto theorem. Assume V is a 
matrix subspace of C™ x " over C (or K). Depending on whether V contains invertible 
elements, the matrix subspace is called either nonsingular or singular [20 . Generically, 
a matrix subspace is nonsingular [22] . Then its subset consisting of invertible elements 
is open and dense [20) . 

Two matrix subspaces V and W are said to be equivalent if W = XVY^ 1 holds 
for invertible matrices X,Y £ C nxn . In view of their properties, equivalent matrix 
subspaces can in many ways be regarded as being indistinguishable. 

Definition 2.1. Suppose V\ and V2 are matrix subspaces of C nx ™ over C (or 
R). Then 

ViV 2 = {V^ : Vi e Vi and V 2 6 V 2 } 

is said to be the set of products of Vi and V2 ■ 

Clearly, we have a homogeneous set, i.e., there holds tViV2 — V1V2 for any 
nonzero scalar t. However, unlike a matrix subspace, the set of products need not 
be closed. (Consider, for example, the LU factorization.) Closedness is certainly of 
importance, e.g., in stable numerical computations @ 

Theorem 2.2. Assume Vi and V2 are matrix subspaces of C nx ™ over C (or R) 
such that V1V2 = if and only if either V\ = or V% = 0. Then V1V2 is closed^] 

Proof. Consider the matrix product 

(Vi,V 2 )^V 1 V 2 

with V\ e Vi and V2 G V2. For any scalars s and t we have (sVi,^) stViV2- 
Thereby, because of the assumptions, we can regard the matrix product as a map 

4 It is well-known that the compution of an LU factorization is not a stable process unless one 
uses partial pivoting. 

5 A natural related problem is as follows. Assume a matrix subspace Vi of C nxn over C (or R) 
is given. Find a matrix subspace V2 of C n x n over C (or R) of the largest possible dimension such 
that the assumptions of Theorem 12.21 arc satisfied. 
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from the product of projective spaces P(Vi) X P(V2) to the projective space P( C" xn ). 
This is a map from a compact space to a compact space. Consequently, by the closed 
map lemma [37J Lemma 4.25], the image of P(Vi) x P(V2) is closed. □ 
For the computation of 

{(Vi.VsOeVi x V 2 : V!V 2 =0}, 

see [10]. 

Example 1 . Let Vi be the set of circulant matrices and V2 the set of diagonal 
matrices in C nxn . By Theorem 12. 2[ then V1V2 is closed. It is readily seen that now 
the matrix product corresponds to the so-called Segre map, a fundamental family of 
functions in algebraic geometry. For the Segre maps, see, e.g., [16] p. 25]. (In other 
words, the Kronecker product of two vectors is really just an instance of the standard 
matrix product restricted to prescribed matrix subspaces.) The above theorem yields 
a natural generalization of such maps. 

Example 2. Certainly, the conditions of Theorem 12.21 are not necessary. For 
instance, the subset of C nx ™ consisting of matrices of rank k at most is closed, a 
property of tremendous importance in approximating with the singular value decom- 
position. It equals the set of products V1V2, where Vi (resp. V2) is the matrix subspace 
of C nx " having the last n — k columns (resp. rows) zeros. 

The following example supports the viewpoint that the set of products yields an 
equally natural "discretization" of Toeplitz operators as Toeplitz matrices do. 

Example 3. For an infinite dimensional noncommutative example, for invertiblc 
Toeplitz operators there is an upper-lower triangular factored Toeplitz structure; see, 
e.g., 25 . For Toeplitz matrices this product does not preserve Toeplitzness. The 
structure remains closed, though. 

Corollary 2.3. Denote by Vi and V2 the subspaces of upper and lower triangular 
Toeplitz matrices of C" xn over C. Then V1V2 is closed. 

Proof. Consider the equation V1V2 — 0. In the product, compute the last row first 
to have either the diagonal of V\ zero or V2 — 0. Then proceed analogously upwards 
by computing next to the last row, to have the claim by using Theorem l2.2l □ 

The set of products in this corollary is intriguing since both subspaces are in- 
vertibleU Thereby it is a straightforward computation to recover if a given matrix 
A € C nxn belongs to V1V2 [20]. The closure of the set of inverses of invertible ele- 
ments is the set of products V2V1, which is closed by analogous arguments. Hence we 
have an elegant symmetry with respect to the inversion 

Although not readily determined, the following quantity appears to be of central 
relevance. To the best of our knowledge, it was initially introduced in [3]. See also 
[TT] for related computations. 

Definition 2.4. Let V be a matrix subspace of C nxn over C (or R). Set 

minrank(V) = min rank(V). 

vev.v^o 



6 Let V and W be two nonsingular matrix subspaces of C nxn over C (or M). If 

{V 1 : V e V n GL(n, C)} = W n GL(n, C), 

then V is said to be invertible |20l . 

7 The problem of characterizing the inverses of Toeplitz matrices has been studied a lot; see, e.g.. 

Eg. 
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This quantity, let us call it the minrank of V, is preserved under equivalence. In 
particular, in the nonsingular case of dimV = 2, the geometric multiplicities of the 
eigenvalues of a matrix determine minrank(V). Namely, let V = span{Vi, V2} with 
Vi, V2 G C nx ™. Suppose X = I and Y is an invertible element of V. Then 

XVY- 1 = span{7, Wi}, (2.1) 

so that it suffices to inspect the eigenvalues of W\ G C" x ™ to determine the minrank 
of V. 

Example 4. The so-called Hurwitz-Radon matrix subspace V of C™ xn over R 
has the property that any nonzero element is a scalar multiple of a unitary matrix. 
Hence, then we have minrank(V) = n. 

Using Proposition 12.21 we can conclude that, whenever 
minrank(Vi) + minrank(V2) > n, 

then V1V2 is closed. 

The set of products need not be somehow curved by the fact that its closure can 
retain the structure of matrix subspace. Of course, we always have a subspace when 
either Vi or V2 is one dimensional. 

Example 5. For a matrix subspace admitting a "factorization" as the product 
of two matrix subspaces, consider the set of (p, g)-band matrices!! Denote by V\ and 
V2 the set of (p, 0)-band and (0, g)-band matrices. Then there exists a factorization 
of a nonsingular A G C" x ™ as A — V1V2 if and only if A is a strongly nonsingular 
(p, g)-band matrix. As strong singularity is a generic property, we can conclude that 
the closure of V1V2 is the set of (p, g)-band matrices, i.e., a matrix subspace. 

Observe that between Examples [2] and [5] there is strong resemblance with regard 
to how the sparsity structures of the matrix subspaces are defined. 

In statistics, the so-called Craig-Sakamoto theorem^ is similarly related with a 
factorization of a matrix subspace; for details on the Craig-Sakamoto theorem, see 
[8]. Besides statistics, this structure is of interest in studying the spectrum in the 
case of three dimensional matrix subspaces |32j . It is, in essence, concerned with the 
claim of the following proposition in the case of k — 2 and dim V\ = dim V2 = 1 • 

Proposition 2.5. Assume a matrix subspace V of C nx ™ over C can be decom- 
posed as V — CI + Y^ij=i %j with matrix subspaces Xj satisfying XjXi = {0} for j < I. 
Then V equals the closure ofY\j =1 (C-I + Xj). 

Proof. Assume V — tl + X\ H h X k G V with Xj G X 3 and a scalar t ^ 0. (If 

t = 0, then V can be approximated with such an element.) By the fact that XjXi = 
for j < I, we can write V = (tl + X X ){I + X 2 /t) •••(/ + X k /t). □ 

The appearing condition on the matrix subspaces Xj forces them to be singular. 
It is noteworthy that this structure is also utilized in computing (inverting) the L 
factor in the LU decomposition of a matrix. 

Let Vi and V2 be nonsingular matrix subspaces of C" x ™ over C of dimension two. 
(Due to the generalized eigenvalue problem, the two dimensional case is possibly the 
most frequently encountered.) In view of the Craig-Sakamoto theorem, it is natural to 



8 A (p, q)-band matrix is a square matrix with lower bandwidth p and upper bandwidth q. 
9 The Craig-Sakamoto theorem: Two real symmetric matrices X\ and X2 satisfy det(7 — tXi — 
sX 2 ) = det(J - tXi) det(J - sX 2 ) for all t, s G K if and only if X 1 X 2 = 0. 
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ask when the closure of Vi V 2 is a matrix subspace. Here we may equally well consider 
equivalent matrix subspaces of the form 

XVi = span{/,Xi} and V 2 Y~ X = span{7, A 2 } (2.2) 

with invertible matrices X,Y E c nxrl chosen appropriately. Then, instead of V1V2, 
it suffices to inspect the set of products XViV^^ 1 ■ 

Theorem 2.6. Let V\ and V 2 be nonsingular matrix subspaces of C™ x " over C 
of dimension 2. If the closure of V\V 2 is a matrix subspace, then it is of dimension 3 
at most. This holds if and only if the matrices in (|2.2[) satisfy 

X 1 (cX 2 - dl) = (aX 2 - bl) (2.3) 

for some constants a,b,c,d E C not all zero. 

Proof. Suppose span{Vf, V 2 } = Vi and span{V3, V4} = V 2 . For the claim we may 
consider the map 

(21, Z2, Z3, 24) i-> Z1Z3V1V3 + Z1Z4V1V4, + Z2Z3V2V3 + Z2Z4V2V4, (2.4) 

from C 4 to C nxn . If the closure of the image were four dimensional, then the set of 
equations 

ziz 3 = bi, Z1Z4 = b 2 , z 2 z 3 = b 3 and z 2 z± = fe 4 (2.5) 

should have a solution for a dense subset of vectors (b±, b 2 , b 3 , 64) of C 4 . Choose 
bi = t + ei, 62 = t + e 2 , 63 = — t + £3 and fe 4 = < + £4 with i 6 C\{0} and small 
ej| <C for j = 1, . . . ,4. (For ei = £2 = £3 = £4 = we clearly have no solutions.) 
We obtain 

Z3 = (1 + e)z 4 and z 3 = (-1 + e)z4, 

where e = ^Enf an d e = e t 3 4 T e e 4 4 ■ Subtracting leads to = (2 + e — e)z 4 which 
is a contradiction for small small e,-. Hence there are no solutions near the point 
(1,1,-1,1). 

Assume the closure of ViV 2 is a matrix subspace. Then the matrices V1V3, V1V4, 
V2V3 and V2V4 are contained in this subspace. (For example, the choices z\ = Z3 = 1 
and Z2 = 24 = yield V1V2, etc.) If the dimension is 3, it follows that V1V3, V1V4, 
V2V3 and V2V4 are linearly dependent, i.e., Xi(X 2 ~ dl) — aX 2 — bi. If the dimension 
is 2, then X\ = aX 2 — bi. 

For the converse, assume dimj^iVs, V1V4, V2V3, V2V4} = 3. (The case of dimension 
2 is trivial.) We may assume, let us say, V1V3 = aiViV^ + a 2 V 2 Vz +C113V2V4 with fixed 
ctj E C, for j = 1, 2, 3. Inserting this into (|2.4I) gives 

(aizi2 3 + ZiZ^ViVi + (a 2 zxz 3 + z 2 z 3 )V 2 V 3 + (a 3 ziz 3 + z 2 z 4 )V 2 V A . 

Let a vector b = (bi,b 2 ,b 3 ) be given. After possibly an arbitrarily small perturbation, 
we may assume b 2 ^ 0. Then necessarily z 3 ^ 0, so that with w — ^ the problem of 
attaining these coefficients is equivalent to finding a solution to the equations 

Zi(eti +w) — ci, a 2 zi + z 2 — c 2 , a 3 z x + wz 2 = c 3 (2.6) 

with c 2 =/= 0. The last two equations, after eliminating z 2 , can be combined into a 
single one £1(0:3 — a 2 w) + c 2 w = c 3 . This combined with the first equation in (12.61) 
results in 

c 2 w 2 + (c 2 Q!i - ci« 2 — c 3 )w + c 2 a 3 - c 3 ai = 
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which should have a nonzero solution satisfying w ^ — a\. This can be achieved, after 
possibly an arbitrarily small perturbations of the coefficients Cj , Ci and C3 . □ 

In particular, the Craig-Sakamoto theorem corresponds to the special case in 
which the constants satisfy a = b = d = and c = 1. 

In the identity (12.3[) we are dealing with what we call a generalized linear frac- 
tional transformation. Namely, if d/c is not an eigenvalue of X2, then X\ is a classical 
linear fractional transformation of X 2 . Combined with the equivalence transforma- 
tions (|2.2[) . this yields a way to construct two dimensional matrix subspaces such that 
the closure of the set of products is a matrix subspace. 

2.2. The set of products and bihomogeneous polynomial maps of bide- 
gree (1, 1). There is a family of polynomial functions, studied especially in algebraic 
geometry, which is intimately connected with the product of two matrix subspaces. 
To describe this connection, for the set of products let us introduce its linearization 
defined as follows. 

Definition 2.7. Assume Vi and V2 are matrix subspaces of C" xn over C (or 
R). The matrix subspace of the smallest possible dimension including V1V2 is said to 
be the linearization 0/V1V2. 

Assume V 1 , . . . , and V- ?+1 , . . . , V k are bases of V\ and V%, respectively. If 
W 1 , . . . , W l is a basis of the linearization W, then the inclusion relation of the lin- 
earization implies that 



VSyt = £ m r Wr 



r=l 

for some constants m* G C. Thereby the product of arbitrary elements V\ G Vi and 
Vi G V2 can be written as 

j k I 

V X V 2 = z » yS WtVt = z T M r wW r (2.7) 

s=l t=j+l r=l 

with z — (zi, ...,Zj) G , w = (wj+i, • • • , u>k) & C' 0- -' and matrices M r — {mf} G 
0x{k-j)_ j n terms of this expansion, the problem converts into inspecting the biho- 
mogeneous polynomial map 



(z, w) 1 — > M(z)w 



z T M 1 



z T Mi 



w (2.8) 



of bidegree (1,1) from x C k ^^ to C . (For computational algebraic geometric 
aspects of such functions, see [10].) Clearly, M ; O — > C ix ( fe -i) j s linear with the 
pth column equaling z T M p at z G C J . 

Example 6. In the proof of Theorem l2.6[ when the closure of V1V2 is the three 
dimensional matrix subspace W, we have j — 2, k = 4 and / = 3. Then W = V1V2, 
W 2 = V2V3 and ^ = ^2^4 with 

Mi = [ a J 1 ],M 2 =[«»o] and Af 3 



The following theorem is of importance. 

Theorem 2.8. Let Vi, V 2 and W fee matrix subspaces of C nxn over C. the 
closure of V1V2 is W, i/ien V1V2 contains an open dense subset ofW. 
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Proof. Using arguments from algebraic geometry, this follows directly from a 
theorem of Chevalley, i.e., from the fact that the map (|2.8p is regular and that its 
image is therefore constructible. (For more details, see, e.g., Theorem 10.2] or 
[17l p. 94].) If the closure of the image (in the standard topology of C') is C ( , then 
C is also the Zariski closure of the image. Thus, it contains a Zariski open set of C l . 
Such a set is open and dense in C. D 

In principle, with (|2.8p the problem of recovering whether a given matrix i£lV 
is factorizable can be approached with the Hilbert Nullstellcnsatz. As opposed to 
considering p.4[) , this problem belongs to the realm of commutative algebra. For the 
existence of a solution, by invoking the effective Nullstellensatz we have the linear 
algebra problem of solving a linear system. It is, however, only formally so because 
of the exponential growth of the size of the linear systems; see Appendix. 

The converse problem is actually of interest. Namely, consider solving a bilinear 
system 

M(z)w = b (2.9) 

with a given b G C . If an expansion (|2.7|) can be established, for some matrix sub- 
spaces Vi, V2 and W, with either Vi or V2 invertible then solving (|2.9[) is straightfor- 
ward with the algorithms proposed in |20j . Consequently, the problem of finding such 
matrix subspaces for a given bilinear system (|2.9p seems to be of central relevance. 

3. Riemannian geometry of the product of matrix subspaces. Next the 
geometry of the set of products is inspected. The flat case is introduced first. Then 
the general case is studied in terms of smooth maps. 

3.1. Factorizable matrix subspaces. The structure appearing in connection 
with the LU decomposition and the Craig-Sakamoto theorem is really just an instance 
of the following general notion of noncommutative factoring. 

Definition 3.1. A matrix subspace W of C nxn over C (or R) is said to be 
factorizable if 

W = ViV2~ (3.1) 
for matrix subspaces V\ and V2 of C" xn over C (or R) satisfying the conditions 

1 < min{dimVi,dim V2} and maxjdim Vi, dim V2} < dimW. (3-2) 

A matrix subspace which is not factorizable is said to be irreducible. As Example 
[5] illustrates, taking the closure may be necessary. Observe though that, for matrix 
subspaces over C, the set of products V1V2 is topologically and measure theoretically 
large in W by Theorem 12.81 

The structure is preserved under equivalence, i.e., W is factorizable if and only if 
XWY- 1 is factorizable for any invertible X, Y e C nxn . 

A central problem, typically tough for one reason or another, is to recover whether 
a given matrix subspace is factorizable. Needless to say, if a matrix subspace W can 
be factored, the interest turns completely on the factors Vi and V^F"! 

Example 7. This is Example [3] continued. In finite dimensions, an analogous 
problem corresponds to asking if the set of Toeplitz matrices is a factorizable matrix 



10 This is manifested by the LU factorization. After computing an LU factorization, the original 
matrix is typically thrown away and only the factors are saved in the storage. 



8 



MARKO HUHTANEN 



subspace. (It is likely of use to recall that Toeplitz matrices are equivalent to Hankel 
matrices.) 

It is noteworthy that when the set of products is closed, the singular elements of W 
are exactly determined by the singular elements of the factors Vi and V2 . (Otherwise 
only an inclusion can be guaranteed.) Hence, then the study of the spectrum of W, 
i.e., its singular elements, reduces to the study of the spectra of Vi and Vi. This is 
precisely the case in the Craig-Sakamoto theorem. 

Example 8. Interpreting the spectrum algebraic geometrically, in the case of 
W = Vi V2 the determinantal variety of W is determined by the determinantal varieties 
of Vi and V2 ■ This "factorization" is intriguing since the determinantal variety of W 
may well be irreducible. For example, we have C nx ™ = V1V2, where Vi = V2 is the set 
of symmetric matrices. (This is a classical result; see, e.g., [TS] and [23] and references 
therein.) 

In general, (|3.ip is a noncommutative notion with respect to the factors. (Cu- 
riously, when dimVi = dimV2 = 1, the commutative case corresponds to the ui- 
commutativity of matrices [18 .) Although more stringent, in group theory there is 
the so-called internal Zappa-Szep product of a group having some aspects in common 
with Definition l3.lF^I For the the internal Zappa-Szep product, see the paper [TJ, 
which is partially expository, and references therein. 

Only the case where the conditions (I3.2j) are satisfied is of interest since otherwise 
we are dealing with the equivalence of matrix subspaces. Consequently, a two dimen- 
sional matrix subspace is never factorizable. Then, through the equivalence (|2.1I) . the 
nonsingular case is completely understood in terms of canonical forms for matrices. 
Two dimensional matrix subspaces are hence fundamentally different and correspond, 
in essence, to classical matrix analysis. Observe that the first nontrivial case, i.e., the 
three dimensional case over C can be regarded as understood by Theorem 12.61 

In Example [2] there is the growth of k and in Example [5] the growth of the 
bandwidth, i.e., we have natural hierarchies of sets of products. The maximum values 
of the indices correspond to the set of products C nxn . Analogously, suppose (|3.1[) 
holds. Take two sequences of nested subspaces 

V\ C V\ C • • • C Vf mVl = Vi and V\ c Vf C • • ■ C V 2 dimV2 = V 2 . 

Then {V[ Vf }j.k yields a natural hierarchy of sets of products such that for the max- 
imum values of the indices the closure of the set of products is W. 

After these general remarks, let us give further illustrations on how Definition l3.il 
actually encompasses numerous classical instances. 

Example 9. Although we are concerned with finite dimensional (noncommu- 
tative matrix) subspaces, infinite dimensional (commutative) problems are certainly 
of equal interest. For the classical problem of integer factoring, let k £ Z be given. 
Associate with k the set fcZ, regarded as subspace of Z over Z. Then the question 
of whether k is a irreducible is equivalent to asking whether fcZ is a factorizable 
subspace. 

For another commutative case, consider the ring of complex polynomials. To this 
corresponds a very classical notion. Namely, denote by Vk the subspace consisting 
of complex polynomials of degree k at most. (Of course, factoring depends heavily 



Let G be a group with the identity element e, and let H and K be subgroups of G. If G = HK 
and H C\ K = {e}, then G is the internal Zappa-Szep product of H and K . 
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on the choice of field.) Then the fundamental theorem of algebra can be stated as a 
factorization result 

V fc = ViVfe-! 

for Vfc. (This can also be formulated as an operator factorization result in infinite 
dimensions for lower triangular Toeplitz operators with finite bandwidth.) Of course, 
this is a tremendously powerful fact with practical implications, i.e., one would cer- 
tainly like to have any given polynomial factored. 

Example 10. Polynomial factoring relates to matrix subspaces as follows. Sup- 
pose A 6 C™ xn and consider 

V = K,(A; I) = span c {I, A, A 2 , . . .}. 

Then, by the fundamental theorem of algebra, V is a factorizable matrix subspace 
whenever deg(A) > 2. 

3.2. Geometry of the product of matrix subspaces. In general, the set 
of products of two matrix subspaces Vi and V 2 of C" x ™ over C (or R) cannot be 
expected to be flat, i.e., to yield a factorizable matrix subspace. The study of its 
geometry involves linear structures, though. To this end, associate with the set of 
products the bilinear map 

*{V 1 ,V 2 ) = V 1 V 2 (3.3) 

from the direct sum Vi x V 2 to C rix ™. Certainly, \& is smooth as it can be treated as 
a bihomogeneous polynomial map once a basis of the vector space Vi x V 2 has been 
fixed appropriately; see (|2.8I) . 

Example 11. Although not done in this paper, let us emphasize that it is 
also natural to study \I/ on subspaces of Vi x V2. For example, take Vi = Vj to be 
the set of lower triangular matrices in C™ xn . Then, related with the LU factoring of 
symmetric matrices, consider ^ on the subspace {(Vi, V2) G Vi x V2 : Vi = V 2 T }- 

If the linearization of V1V2 is not the whole C nXTl , then the set of products is 
said to be degenerate. In view of this, for matrix subspaces of C" x ™ over C, a 
generalization of Picard's theorem [H] can be used to bound the dimension of the 
linearization. For this, assume A G C rix ™ and look at the map A — defined as 

(V 1 ,V 2 )^A-V 1 V 2 (3.4) 

If A V1V2, then this does not have zeros and hence we may apply the following 
theorem with m — dim Vi + dim V2 and I — n 2 ~ 1. 

Theorem 3.2. JTffi Let f : C m — > P;(C) be a holomorphic map that omits 
I + k hyyervlaneW^ in general position, k > 1 . Then the image of f is contained in 
a projective linear subspace of dimension < [l/k], where the brackets mean greatest 
integer. 

To inspect the structure of the set of products more locally, on any matrix sub- 
space V of C nx ™ over C the standard inner product 

(Vi, V 2 ) = tr(y 2 *Vi) with Vi, V 2 e V (3.5) 

12 A hyperplane in homogeneous coordinates 20, . . . , z\ is {(20, • ■ ■ , z{) : 5Zj=o a i z i = 0} f° r fixed 
aj G C. 
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is used. (Take the real part for matrix subspaces over K.) The respective norm || • ||f 
is the Frobenius norm. To proceed Riemannian geometrically, take a smooth curve 
with the Taylor expansion 

c(t) = (V 1 ,V 2 ) + t(W u W 2 ) + t 2 (U 1 ,U 2 ) + ■ ■ ■ 

in Vi x V 2 passing through a point (Vi, V 2 ). Then \& maps this curve to C" xn as 

V X V 2 + t(ViW 2 + W X V 2 ) + t 2 (V x XJ 2 + W X W 2 + UiV 2 ) + ■■■ . (3.6) 

Considering its linearization, the linear terms span the matrix subspace 

VxV2 + ViV 2 . (3.7) 

In particular, the dimension of this subspace yields the rank of \1/ at (Vi, V 2 ). 

It is clear that the maximum rank yields a lower bound on the dimension of the 
linearization of V\V 2 . (Proof: fl3.7[) is a subset of the linearization of V\V 2 .) 

PROPOSITION 3.3. Let Vi andV 2 be two subspaces of C nxn over C (orR). Then 
V1V2 equals its linearization if and only ifViV 2 contains the matrix subspace (|3.7[) for 
any (V\,V 2 ). 

Proof. Suppose V1V2 is a subspace, i.e., equals its linearization. Since $ : Vi x 
V2 — > V1V2, it follows that V1V2 contains the matrix subspaces (|3.7p . 

Suppose V1V2 contains the matrix subspaces Q3.7p . Then ViY + XV 2 £ V1V2 for 
any Vi, X £ Vi and Y. V\ £ V2, i.e., it contains the sums. The multiples are contained 
by the homogeneity. Thereby V{V 2 is a subspace. □ 

The following facts are well known; see, e.g., Chapter 2], 

Proposition 3.4. The rank of '\& generically attains the maximum. 

Proof. Consider the derivative of 'J regarded as a function of m = dim Vi + dim V 2 
variables after fixing bases of Vi and V 2 . Recall that a matrix has rank r if and only 
if there exists at least one non-zero r-by-r minor. Computing the determinant of the 
respective minor of the derivative gives a nonzero polynomial in m variables. The 
points where the maximum of the rank is not attained belong to its zero set. □ 

The following also justifies calling ()3.7|) locally the tangent space of V1V2 at 
V(V 1 ,V 2 ) = V 1 V 2 . 

Proposition 3.5. Let the rank of attain the maximum at {Vi,V 2 ). Then the 
image of a neighbourhood of(Vi,V 2 ) under $ is a smooth submanifold of C™ xn of the 
dimension equaling the rank. 

Proof. Let m = dim Vi + dim V 2 and denote by k rank of ^ at (Vi, V 2 ). Because 
of the constant rank theorem, we have the normal form 

if) o * o (fr x {xx, . . . ,x m ) H> (xi,...,x k ,0,0, ...,0) 

for appropriate charts <f> and ip. Denote the components of ip by ipj and consider the 
map (V'fe+i) • ■ • , VVi 2 ) fromC" XTl to C™ 2 ~ fc . For this map, look at the inverse image of 
(0, 0, ... , 0) to have the claim (a consequence of the constant rank theorem). □ 

Assume the rank of 'J attains its maximum at (Vi, V 2 ). By the above proposition, 
locally the image can be regarded as a smooth submanifold of C" xn . With respect to 
the standard inner product on C nx ", denote by Py^y^ the orthogonal projector on 
C" x " onto the tangent space V1V2 + ViV 2 . Then to Riemannian geometrically assess 
how curved V1V2 is at V 2 ), consider the map 



Q{ Vl ,v 2 )(W u W 2 ) = 2(L-P VuV2 )W 1 W 2 



(3.8) 
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on Vi x V2. Using this notation, its introduction can be argued as follows. 

Proposition 3.6. Let the rank of *S> attain its maximum at (Vi,^). Then, 
for the image of a neighbourhood of (Vi, V2) under "J, the extrinsic curvature of the 
geodesic passing through \&(Vi, V2) with the speed vector V\Wi + W1V2 equals 



Proof. An application of (|3.8p always corresponds to a curve p.6[) having the 
acceleration orthogonal to V1V2 + V1V2 which is achieved by choosing the coefficient 
(1/1,1/2) such that V1U2 + W\W% + U1V2 belongs to the orthogonal complement of the 
matrix subspace V\ V2 + V1V2. Such a property is required from the geodesies passing 
through ^(Vi,V2); see (251 pp. 138-139]. Hence, for a geodesies passing through 
^(Vi, V2) with the speed vector V1W2 + W1V2, the coefficient (1/1,1/2) is determined 
by this condition. □ 

If the map (13.81) vanishes identically, then the set of products V1V2 belongs to 
V1V2 + V1V2. Then also the linearization of V1V2 equals this matrix subspace. 

Geodesies and measuring curvature are closely related. Namely, the second fun- 
damental form II for the image of $ in a neighbourhood of (V\, V2) can be found by 



setting UQ(v u v 2 )(Wi + Wi,W 2 + W 2 ) - Q (Vl ,v 2 )(W u W2) - Q( Vl ,v 2 )(W u W 2 )) to be 
ll(W 1 ,W2,W 1 ,W 2 ) = (I -Pv 1 ,v 2 )(W 1 W2 + W 1 W 2 ). 

For more details on the second fundamental form and its geometric interpretation, 
see [261 P- 138]. 



As a first example, Theorem 1 2 . 61 corresponds to vanishing curvature. For another, 
with nonvanishing curvature, consider the subset of matrices related with approximat- 
ing with the singular value decomposition. 

Example 12. This is Example [2] continued, i.e., consider /F k C C nx ", the set 
of matrices of rank k at most. Take (Vi, V 2 ) to be a generic point of Vi x V2 and 
let V) = U1T11W1 and V 2 = L^^W^* b e t ne singular value decompositions of V\ and 



where W is the matrix subspace of C" XTl consisting of matrices whose first k rows 
and columns can be freely chosen. Hence, the points (Wi, W2) G V\ x V2 satisfying 
Q(Vi,v 2 )(Wi,W 2 ) = 2W1W2 are readily determined. Consequently, V1V2 = /Fk can be 
regarded as being maximally curved when measured in terms of (|3.9p . It is of interest 
to note that V2V1 is flat. 

Observe that although Q(V!,v 2 ) yields a local measure of curvature, for the image 
of ^ around the point ^(V\, V2), it possesses a global character by the fact that the 
appearing orthogonal projector operates on the full image of iff. This provides an 
opportunity to use local information to draw conclusions about global properties of 
the set of product as follows. 

Theorem 3.7. Let V\ and V2 be matrix subspaces of C nxn over C. Then V1V2 
equals its linearization if and only if ()3.8p vanishes for some (Vi, V2). 

13 For j = 1,2, the matrices Uj £ C nx " and Wj £ C™ x ™ are unitary. In the matrices £j only 
the first k diagonal entries can be nonzero. For more details on the singular value decomposition, 
see |13| . 




(3.9) 




v 1 v 2 + v 1 v 2 = u 1 vvw 2 *, 
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Proof. Assume ViV 2 equals its linearization W. By Sard's theorem, the set of 
critical values of \t is of the first category with Lebesque measure zero; see, e.g., [301 
p. 260, Theorem 1]. However, by Theorem 12. 81 the image contains an open subset of 
W. Its Lebesque measure is obviously positive. Thereby there must be points in the 
image which are regular values. Consequently, (|3.8p vanishes for some (Vi,V 2 ). 

For the converse, suppose p.8[) vanishes for some (Vi,V 2 ). Clearly, then the clo- 
sure of V1V2 belongs to the respective matrix subspace W = V1V2 + ViV 2 , i.e., the 
linearization of V\V 2 . We may regard W : Vi X V2 — > W. By the constant rank theo- 
rem, the image of a neighbourhood of (Vi, V2) under 'J contains an open set (in the 
standard topology) in W. In the Zariski topology, the image of ^ is constructible 
and contains an open subset of its closure. This follows from a theorem of Chevalley. 
(For more details, see, e.g., [28l Theorem 10.2] or [TT1 p. 94].) Because the image of 
\l/ contains an open subset in the standard topology, the open subset of the image of 
in the Zariski topology is dense in W. □ 

Obviously, under the assumptions of this theorem, W = V1V2 is factorizable 
(assuming (13. 2[) is satisfied). 

An immediate example is given by the LU factorization. 

Example 13. Using Theorem 13.71 it is straightforward to conclude that an LU 
factorization exists for elements in a dense open subset of C nxn . (Of course, this 
is well known; a nonsingular matrix can be LU factored if and only if it is strongly 
nonsingular P23 p. 162].) Namely, let Vi denote the set of lower triangular matrices 
and V2 upper triangular matrices with constant diagonal. Then at (I, I) £ Vi x V2 the 
rank of ^ is readily seen to be n 2 . Consequently, (|3.8|) vanishes, so that C™ x " = V1V2. 

Of course, we may view this example just as a special case of the following con- 
sequence of Theorem 13. 71 

Corollary 3.8. Suppose Vi and V2 are matrix subspaces of C™ x " over C both 
containing the identity. If C nxn = Vi + V 2 , then C nxn = 

Analogously can be formulated claims for lower (upper) triangular matrices in 
case of subspaces Vi and V2 of lower (upper) triangular matrices as follows. 

Corollary 3.9. Suppose W is equivalent to a subalgebra of C nx ™ over C 
containing invertible elements. Then W is factorizable. 

Proof. We may assume W is a subalgebra of C nx ™ over C containing invertible 
elements. Observe that W contains the identity. To see this, assume A £ W is 
invertible. Then take a polynomial satisfying p(A) — A^ 1 . We have / = Ap(A) £ W. 

Denote by d the dimension of W. Take a basis W\, . . . , Wd of W and set Vi = 
span{7, W u W 2 , . . . , W k } and V 2 - span{7, W k+1 ,W k+2 , W d }. Clearly, V1V2 C 
W. At (I, I) the rank of * is d. □ 

For a classical factorization, any matrix A £ c™ x ™ j s the product of two sym- 
metric matrices such that there are at least n degrees of freedom to construct a 
factorization [20]. (See also Example [H) This redundancy can be reduced as follows. 

Proposition 3.10. Let Vi be the set of symmetric matrices and V2 the subset of 
symmetric matrices having constant antidiagonal. Then C nx,i = ViV 2 . 

Proof. Suppose n is odd and denote by J the permutation matrix having ones 
on its antidiagonal. Then, for a generic diagonal matrix D = diag(rfi, dz, . . . , d n ), 
the rank of \& at (D, J) € Vi X V 2 can be shown to equal n 2 . (For n even, proceed 
similarly with (J, D) £ V\ x V2O To see this, consider DV 2 + V\ J. Clearly, the (j, k) 
entry of V\ J is the (j, n — k + 1) entry of V\. For k ^ j and k ^ n — j + 1 this means 
that the entries in DV 2 + V\ J located symmetrically with respect to the diagonal 
and antidiagonal are interdependent, i.e., the (J, k), {k,j), (n — k + 1, n — j + 1) and 



PRODUCT OF MATRIX SUBSPACES 



13 



(n—j + l,n — k + l) entries. To satisfy DV2 + V\ J — C nxn yields us the linear system 



dj 
d k 

d n 
d„ 



10 
1 

-fc + i 1 
1 



V 2 (j,k) 
V 2 (n-k+l,n-] + 
Vi(i,n-fc+l) 
Vi(fe,n-j+l) 



M{j,k) 

M(k,j) 
M(n-k+l,n-j+l) 
M(n-j + l,n-k+l) 



which should have a solution for any right-hand side. This is possible if and only if 
dkd n -k+i q £ Qr j y, anc j ^ n _ j _|_ ^ _ -p Qr k = j an d = rt - j + 1 we 



1 



obtain the condition ci,- ^ 0. These are the genericity conditions the matrix I? needs 
to satisfy. □ 

Since Vi is an invertible matrix subspace, the factorization of the proposition can 
be computed with the methods proposed in [20] . 

4. Conclusions. Motivated by factorization problems, the set of products of 
two matrix subspaces is studied. Differential geometric approach applied to the re- 
spective smooth mapping yields a measure of curvature for the set of products. Its 
vanishing corresponds to the concept of factorizable matrix subspace. The notion 
of irreducible matrix subspace was introduced. The LU factorization, the singular 
value decomposition and the Craig-Sakamoto theorem served as illustrative examples 
how seemingly different concepts (algorithmically at least) can be put under the same 
caption. 

Appendix: Hilbert Nullstellensatz. Let P\,...,Pk be complex polynomials 
in n variables. There is no common zero in C™ if and only if there are complex 
polynomials qj in n variables satisfying 



This is the Hilbert Nullstellensatz. The effective Hilbert Nullstellensatz states that 
the degrees of the qj may be assumed to satisfy 

f D n if D > 3 
deg qj < < 2min{n,fc} if _D = 2 ' 



where D = max deg pj. See [2], [53] and [3"T] . 
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